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Research has focused on increasing the treatment integrity of school-based interventions by 
utilizing performance feedback. The purpose of this study was to extend this literature by 
increasing special education teachers’ treatment integrity for implementing antecedent and 
consequence procedures in an ongoing behavior support plan. A multiple baseline across 
teacher-student dyads (for two classrooms) design was used to evaluate the effects of 
performance feedback on the percentage of antecedent and consequence components 
implemented correctly during 1-hr observation sessions. Performance feedback was provided 
every other week for 8 to 22 weeks after a stable or decreasing trend in the percentage of 
antecedent or consequence components implemented correctly. Results suggested that 
performance feedback increased the treatment integrity of antecedent components for 4 of 5 
teachers and consequence components for all 5 teachers. These results were maintained following 
feedback for all teachers across antecedent and consequence components. Teachers rated 
performance feedback favorably with respect to the purpose, procedures, and outcome, as 
indicated by a social validity rating measure. 
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Treatment integrity, the implementation of 
an intervention as intended, is a topic of interest 
among researchers and change agents (Gresham, 
Gansle, & Noell, 1993; Watson, Sterling, & 
McDade, 1997). This interest in treatment 
integrity is accompanied by a recent focus on 
accountability by school systems with the 
enactment of No Child Left Behind (U.S. 
Department of Education, 2001). Con- 
sequently, researchers have attempted to iden- 
tify parsimonious, effective, and time-efficient 
strategies to ensure the integrity of school- 
based interventions (Cossiart, Hall, & Hop- 
kins, 1973; Gillat & Sulzer-Azaroff, 1994; 
Mortenson & Witt, 1998; Noell, Duhon, 
Gatti, & Connell, 2002; Noell, Witt, Gil- 
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bertson, Ranier, & Freeland, 1997; Witt, Noell, 
LaFleur, & Mortenson, 1997). One successful 
intervention strategy for enhancing treatment 
integrity is performance feedback (Mortenson 
& Witt, 1998; Noell et al., 1997, 2000, 
2002; Witt et al., 1997). In the literature, 
performance feedback has encompassed several 
components including (a) review of data, (b) 
praise for correct implementation, (c) corrective 
feedback, and (d) addressing questions or 
comments. 

Performance feedback has been used in 
regular education settings to improve the 
implementation of academic interventions 
(Mortenson & Witt, 1998; Noell et al., 2000; 
Witt et al., 1997), peer tutoring (Noell et al., 
2000), the use of contingent praise 0ones, 
Wickstrom, & Friman, 1997; Martens, 
Hiralall, & Bradley, 1997), and implementa- 
tion of behavior-management interventions 
(Noell et al., 2002). Noell et al. (2002) 
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examined the integrity with which 4 general 
education teachers employed interventions to 
decrease out-of-seat behavior and talking out in 
8 elementary school students. Integrity was 
measured by permanent products that consisted 
of data collected either by students’ self-report 
or by teachers. This study represents an 
example of the effects of performance feedback 
on one aspect of behavior interventions: data 
collection. In other situations, it may to be 
important to investigate integrity of specific 
intervention components for complex individ- 
ualized behavior support plans. 

Two other aspects of plan implementation 
that deserve consideration are (a) what ante- 
cedent interventions are implemented to de- 
crease the likelihood of problem behavior and 
(b) the teacher’s response to students’ target 
behaviors. Teachers may not accurately imple- 
ment both types of procedures with equal inte- 
grity because implementation of antecedent 
and consequence procedures may require 
different skills. Implementation of antecedent 
procedures requires teachers to plan and pre- 
vent the occurrence of behavior (Luiselli & 
Cameron, 1998), whereas consequence proce- 
dures require teachers to impose contingencies 
after the occurrence of problem behaviors 
(Kern, Choutka, & Skol, 2002). 

The frequency and structure of performance 
feedback have been a focus of several studies. 
For example, research has demonstrated that 
weekly feedback (Mortenson & Witt, 1998) 
leads to increases in treatment integrity and 
may be more practical for supervisors, clin- 
icians, and consultants than daily performance 
feedback. The schedule and immediacy of 
performance feedback may also influence the 
maintenance of treatment integrity. For exam- 
ple, in many studies performance feedback 
has consisted of reviewing data from the 
previous day (Noell et al., 1997, 2002; Witt 
et ah, 1997) or week (Mortenson & Witt, 
1998), rather than providing feedback on the 
same day. 


The purpose of the present investigation was 
to extend the current research on treatment 
integrity by (a) directly examining the effects of 
performance feedback on special education 
teachers’ implementation of antecedent and 
consequence procedures of ongoing individual- 
ized behavior support plans, (b) administering 
performance feedback every other week and 
during the same day as the observation, and 
(c) assessing short-term maintenance effects of 
performance feedback. 

METHOD 

Participants and Setting 

The present study was conducted in a private 
school for students with acquired brain injury 
who exhibited significant behavior problems. 
Students were all male and ranged in age 
from 10 to 19 years. Three of the students 
had nontraumatic acquired brain injuries, and 2 
students had been diagnosed with a traumatic 
brain injury. The sample of students observed 
comprised approximately 10% of the total 
population of the school. 

Observational data were collected in two 
special education classrooms on a total of 5 
teacher-student dyads. Two dyads (i.e., Mr. 
Canton and Seth; Mr. Martin and Philip) were 
identified from Classroom 1, and 3 dyads (i.e., 
Ms. Lowell and Brian; Ms. Malden and Jason; 
Mr. Mack and Darrin) were identified from 
Classroom 2. All teachers participating in the 
study had earned a bachelor’s degree and were 
enrolled in a master’s level program in special 
education. Teachers’ experience working in this 
environment ranged between 6 and 30 months. 

Materials 

Behavior support plans. Individualized be- 
havior support plans were previously created for 
each student and were ongoing at the time of 
the investigation. The plans had been in place 
for an average of 4 months at the time of the 
study. These plans consisted of individualized 
multicomponent interventions that prescribed 
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both antecedent and consequence procedures 
for students’ targeted problem behaviors. 
Teachers were expected to employ these be- 
havior support plans under specific conditions 
(e.g., activities, times of day, and contingent on 
student behaviors). Six behaviors were targeted 
for Seth (tantrums, perseverative speech, in- 
appropriate speech, inappropriate social behav- 
ior, teasing, and noncompliance), Philip (major 
aggression, minor aggression, inappropriate 
speech, inappropriate touching, public expo- 
sure, and instigation), and Brian (aggression, 
destruction, peer instigation, inappropriate 
speech, inappropriate sexual behavior, and 
noncompliance). Four behaviors were targeted 
for Jason (tantrums, inappropriate speech, in- 
appropriate social behavior, and invasion of 
space), and six behaviors were targeted for 
Darrin (physical aggression, property destruc- 
tion, wandering, inappropriate verbalizations, 
mimicking, and noncompliance). 

Integrity data sheet. The integrity data sheet 
was three pages long and included (a) the type 
of procedure (i.e., antecedent or consequence) 
that was prescribed for that particular student, 
(b) an operational definition of each compo- 
nent of the intervention (copied directly from 
each student’s behavior support plan), (c) 
observer ratings of the teacher implementation 
of each component (see below), and (d) a space 
for the observer to record comments or 
examples. Each component consisted of specific 
behaviors that teachers were instructed to 
engage in, either as a result of the activity a 
student was required to perform or in response 
to students’ behavior. For example, when asked 
to change from a preferred to a nonpreferred 
activity, Ms. Malden was instructed to provide 
Jason with three transition warnings (i.e., at 
2 min, 1 min, and 30 s). Another example of 
an antecedent component specified that Mr. 
Mack explain all expectations to Darrin using 
first-then statements for every activity that he 
engaged in. Consequences included reinforce- 
ment such as receiving social attention (e.g., pat 


on shoulder, praise, conversation) contingent 
on appropriate social behavior (e.g., sharing, 
complimenting peers) or planned ignoring 
following the occurrence of inappropriate 
speech. Table 1 lists all the antecedent 
and consequence components that were pre- 
scribed in various students’ behavior support 
plans. 

The integrity data sheet divided each plan 
into 10 to 13 individualized components (see 
the Appendix for Brian’s integrity data sheet; 
other participants’ data sheets are available from 
the authors on request). One of three levels of 
implementation integrity was scored under the 
implementation rating section of the integrity 
data sheet: (a) implemented as written (i.e., the 
entire component was implemented every time 
the target behavior occurred or when the 
situation required an antecedent component), 
(b) not implemented as written (i.e., sometimes 
implemented the entire component as written, 
implemented part of the component as written, 
or did not employ the component as written), 
and (c) no opportunity to observe (i.e., the target 
behavior did not occur, or the antecedent was 
not present). For example, Ms. Malden received 
a rating of implemented as written if she 
provided all three transition warnings every 
time Jason was required to change from 
a preferred to a nonpreferred activity during 
the observation period. However, she received 
a rating of not implemented as written if she 
provided the three warnings for only two of 
three opportunities that Jason had to change 
from a preferred to a nonpreferred activity or 
if she provided only two transition warnings. 
Mr. Mack earned a rating of implemented as 
written if he blocked Darrin from leaving the 
classroom, redirected him to the scheduled 
activity, and provided a warning that Darrin 
would lose a penny for every time that he 
attempted to elope from his classroom. If Mr. 
Mack did not carry out all parts of this 
component every time that Darrin attempted 
to elope from his classroom, he received a rating 
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Table 1 

Behavior Support Plan Components Classified as Antecedents and Consequences 


Classroom 

Antecedents 

Consequences 

i 

Transition warnings 

Self-monitoring 


Explicit instructions 

DRA 


Modification of presentation of demands 

Time-out 


Breaks incorporated into work periods 

Planned ignoring 


Choices 

Ignore and redirect 


Scheduled activities (limit down time) 

Guided compliance 


Modeling 

Contingent protective hold 

Contingent escort 

2 

Schedule of preferred and nonpreferred tasks 

DRO 


Transition warnings 

DRA 


Explicit instructions 

Planned ignoring 


Modification of presentation of demands 

Ignore and redirect 


Choice 

Block and redirect 


Preteaching 

Guided compliance 


Breaks incorporated into work periods 

Response cost 


One-to-one attention 

Contingent protective hold 


Seating with few distracters 

Contingent escort 


Note. DRO = differential reinforcement of other behavior; DRA = differential reinforcement of alternative behavior. 


of not implemented as written. In the comments 
and examples section of the integrity data sheet, 
consultants could provide an example of how 
the teacher implemented the component. 

Percentage of correct implementation was 
used as the measure of treatment integrity, and 
was calculated by dividing the the number of 
plan components that were implemented as 
written divided by the total number of plan 
components teachers had the opportunity to 
exhibit. The average percentage of the plan 
components that teachers had the opportunity 
to exhibit was 61% (range, 50% to 90%) for 
Classroom 1 and 57% (range, 40% to 77%) for 
Classroom 2 across teacher-student dyads. 
There were no opportunities for teachers to 
exhibit all the components in the behavior 
support plan in any single observation period. 
It should be noted that consultants took notes 
on specific deviations from the written pro- 
cedure. 

Social validity. A 10-item questionnaire that 
examined the acceptability of the performance 
feedback and corresponding integrity observa- 
tions was administered to each teacher after 
performance feedback was terminated. Three 
items addressed the purpose of performance 
feedback, four items inquired about the 


procedures of the observation and feedback 
sessions, and three items asked teachers to rate 
the outcomes of the feedback sessions (e.g., was 
it helpful to increase integrity of plan imple- 
mentation) (Schwartz & Baer, 1991). Respon- 
dents were required to rate each item on a Likert 
scale ranging from 1 ( strongly disagree ) to 5 
{strongly agree). Teachers were instructed not to 
place their names on the questionnaires, and 
a box was placed in a common area for teachers 
to return the forms. 

Training 

All teachers had received both general and 
student-specific formal training in implement- 
ing behavior support plans prior to this 
investigation. After being hired at the school, 
teachers received 4 hr of training in basic 
principles of applied behavior analysis. 
Teachers also received ongoing in-service train- 
ing on general behavior principles four times 
annually. One in-service session was conducted 
during the course of this project, and the topic 
(i.e., task analysis) was unrelated to these 
students’ behavior support plans. Teachers 
received 2 weeks of specific training on individ- 
ual students’ plans at the time they were 
developed. This training included reviewing 
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the written plan with a consultant (i.e., one of 
the authors), modeling by the consultant in the 
classroom with the target student, prompting 
the teacher-in-training to employ the compo- 
nents as written, and immediate performance 
feedback given by the consultant. Teachers were 
not trained to a standardized criterion; howev- 
er, training continued until they could verbally 
report each component of the plan and 
reported that he or she could implement the 
plan. In addition, consultants were present in 
the classrooms on a daily basis and provided 
informal feedback to teachers. 

Procedure 

Observation sessions. Observations were con- 
ducted an average of every 2.1 weeks for each 
dyad (range, 1 to 3 weeks). Variations in the 
time between observation sessions reflected 
teachers’ earned leave time and school breaks. 
Each observation was planned to be 60 min 
long. There were five occasions on which 
observations were less than 60 min (range, 45 
to 55 min). The length of time for each 
observation session and time between observa- 
tion sessions remained consistent throughout 
the baseline, intervention, and maintenance 
phases of data collection. As recommended by 
Gresham et al. (1993), we attempted to reduce 
reactivity by conducting observations on a vari- 
able-time schedule and used consultants who 
routinely worked in the classrooms as primary 
observers. The consultant assigned to Class- 
room 1 spent 50% of his total time conducting 
observations. Nine percent of the consultant’s 
time in Classroom 1 was devoted to this 
project. The consultant assigned to Classroom 
2 devoted 33% of her time to conducting 
observations. Six percent of the consultant’s 
time in Classroom 2 was devoted to this 
project. 

Baseline. Baseline consisted of observing 
each student-teacher dyad and completing the 
integrity data sheet without the teachers’ 
knowledge of the observation. Feedback was 
provided following the observation. Teachers 


were not told the purpose of the observation 
during baseline. This was not unusual, because 
observations were a consistent part of the duties 
of experimenters. 

Intervention. Performance feedback was im- 
plemented after stable or decreasing perfor- 
mance in baseline was demonstrated by either 
the percentage of antecedent components or 
consequence components implemented as writ- 
ten. The first feedback session was provided to 
each teacher after the last baseline session to 
address inadequate plan implementation as 
soon as possible for the benefit of the student. 
On the same day as each observation, the 
experimenter spent an average of 12 min with 
the target teacher outside the classroom. At this 
time, components in the behavior support plan 
were reviewed, and feedback was provided on 
all the components that were observed. 
Feedback included providing praise for compo- 
nents followed as written and constructive 
feedback for those components that were 
followed sometimes or not at all. Constructive 
feedback consisted of reviewing the specific 
components observed and explaining how the 
component should have been implemented. 

Performance feedback was terminated after 
improved performance had stabilized. Two 
maintenance sessions were conducted for Mr. 
Canton and Mr. Martin, three maintenance 
sessions were conducted for Ms. Lowell and 
Ms. Malden, and one maintenance session was 
conducted for Mr. Mack. Maintenance sessions 
were identical to baseline sessions. The first 
maintenance session occurred 5 weeks after the 
last feedback session. Each subsequent follow- 
up session occurred at 5-week intervals across 
dyads. 

Experimental design. A concurrent multiple 
baseline across teacher-student dyads design 
was selected for each classroom to evaluate the 
efficacy of performance feedback. Maintenance 
sessions were employed to examine the short- 
term effects of the intervention when feedback 
was no longer provided. 
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Interobserver Agreement 

Interobserver agreement was collected across 
20% of the sessions. A second independent 
observer was present and used the same 
treatment integrity form to observe the teachers 
directly. On a component-by-component basis, 
comparisons were conducted between the 


second observer and the primary observer’s 
responses. Agreement was calculated by di- 
viding the number of agreements per compo- 
nent by the number of agreements plus 
disagreements per component and multiplying 
by 100%. Mean agreement across dyads was 
95% (range, 91% to 100%). 


Performance Feedback Follow-up 



Sessions 


Figure 1. Percentage of antecedent and consequence components implemented as written across teacher-student 
dyads for Classroom 1. 
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RESULTS 

Teacher Implementation of the Behavior 
Support Plan 

Classroom 1. Figure 1 presents the percent- 
age of antecedent and consequence compo- 
nents implemented as written (correctly 
implemented) for 2 student-teacher dyads in 
Classroom 1. During baseline, a decreasing 
trend was evident for the percentage of con- 
sequence components that Mr. Canton imple- 
mented as written [M = 31%). Performance of 
antecedent components implemented correctly 
was stable ( M — 69%) during baseline. 
Following performance feedback there was an 
immediate increase in the percentage of 
consequence components correctly implemen- 
ted ( M — 92%) and a gradual increase in the 
percentage of antecedent components correctly 
implemented ( M — 92%). 

During baseline, Mr. Martin’s performance 
was variable for the correct implementation of 
antecedent and consequence components. Mr. 
Martin correctly implemented 100% (eight 
sessions), 50% (three sessions), or none (two 
sessions) of the consequence components dur- 
ing baseline, which makes these data difficult to 
interpret. The mean percentage of components 
correctly implemented during baseline was 
85% for antecedents and 73% for conse- 
quences. Following performance feedback, 
Mr. Martin demonstrated 100% correct imple- 
mentation across consequence components for 
three consecutive sessions ( M — 100%). There 
was little change in Mr. Martin’s correct 
implementation of antecedent components 
(M = 89%). 

Classroom 2. Figure 2 presents the percent- 
age of antecedent and consequence components 
implemented as written for 3 student-teacher 
dyads in Classroom 2. During baseline, a de- 
creasing trend was evident for the percentage of 
antecedent components correctly implemented 
by Ms. Lowell {M — 40%). Ms. Lowell did 
not correctly implement any consequence com- 
ponents during baseline ( M = 0%). Following 


performance feedback, substantial improve- 
ments were demonstrated in the percentage of 
correctly implemented antecedent ( M — 78%) 
and consequence components ( M — 95%). 

During baseline, Ms. Malden’s correct 
implementation of antecedent components 
was stable ( M = 47%). Correct implementa- 
tion of consequence components appeared to 
be increasing in baseline ( M — 27%). Every 
attempt was made to have stable or decreasing 
trends in baseline prior to implementation of 
performance feedback. Llowever, a clinical de- 
cision was made to employ performance 
feedback when the implementation of anteced- 
ent components was stable. Following perfor- 
mance feedback, correct implementation of 
antecedent ( M — 95%) and consequence 
( M = 95%) components improved. 

The sequence of baseline and performance 
feedback was varied for Mr. Mack and Darrin. 
During the initial baseline phase, Mr. Mack 
displayed stable performance for the imple- 
mentation of antecedent components as writ- 
ten. Mr. Mack did not correctly implement any 
consequence components. Following the initial 
baseline, an increase in the frequency and inten- 
sity of Darrin’s physical aggression required 
several revisions in his behavior support plan. 
Following Revision 1, there was an increase in 
the correct implementation of consequence 
components, and antecedent performance 
remained at baseline levels. Revision 2 encom- 
passed both baseline and performance feedback. 
Performance feedback resulted in an increase in 
the percentage of antecedent and consequence 
components correctly implemented. Improve- 
ments in performance were maintained during 
Revision 3 across antecedent and consequence 
components. 

Maintenance observations. Treatment integ- 
rity was maintained at high rates during the 
follow-up phase across all teachers (Figures 1 
and 2). The percentages of antecedent ( M — 
83%) and consequence (M — 100%) compo- 
nents correctly implemented were maintained 


212 


ROBINS. CODDING et al. 


Baseline 


Performance Feedback 


Follow-up 


<D 

CO 

"o 

C L' 


<t> 

CL 



Sessions 


Figure 2. Percentage of antecedent and consequence components implemented as written across teacher— student 
dyads for Classroom 2. 


5 and 10 weeks following the termination of 
performance feedback for Mr. Canton. For Mr. 
Martin, high levels of correct implementation 
were maintained across antecedent (M — 83%) 


and consequence (M = 100%) components 5 
and 10 weeks after feedback. For Ms. Lowell, 
high percentages of correctly implemented 
antecedent [M — 89%) and consequence 
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(M = 92%) components were maintained at 5, 
10, and 15 weeks without performance feed- 
back. Ms. Malden also implemented high 
percentages of antecedent (M — 92%) and 
consequence (M — 100%) components as 
written during three follow-up observations 
(i.e., 5, 10, and 15 weeks after intervention). 
Five weeks after performance feedback was 
terminated, Mr. Mack correctly implemented 
100% of antecedent and consequence com- 
ponents. 

Social Validity 

Average ratings across all teachers ranged 
from 4.5 to 5.0 for each item. For the purpose 
of feedback sessions, intervention procedures, 
and intervention outcomes, mean ratings of 
4.8, 4.7, and 4.9, respectively, were obtained, 
indicating that teachers strongly agreed with 
items related to the purpose and procedures as 
well as the benefits of the intervention out- 
comes on their skills and the subsequent impact 
on their students. 

DISCUSSION 

The results revealed that accurate implemen- 
tation of ongoing behavior support plans 
improved across 5 student-teacher dyads fol- 
lowing performance feedback. This research 
extends the literature by demonstrating the 
efficacy of performance feedback on the 
treatment integrity of individualized multicom- 
ponent behavior support plans in a special 
education setting. This study also showed 
that the results of performance feedback were 
maintained for up to 15 weeks. In contrast to 
other studies (Mortenson & Witt, 1998; 
Noell et al., 1997, 2000; Witt et al., 1997), 
treatment integrity was assessed using direct 
observation, and performance feedback was 
provided every other week rather than daily 
(Noell et al., 1997, 2000; Witt et al., 1997) 
or weekly (Mortenson & Witt, 1998) and 
on the same day that the observation 
occurred. 


Performance feedback resulted in greater 
percentages of both antecedent and conse- 
quence components correctly implemented for 
4 of 5 teachers. For the 5th teacher (Mr. 
Martin), performance feedback resulted in 
increases in correct implementation of conse- 
quence components. These data also illustrate 
that during baseline, 4 of the 5 teachers 
correctly implemented more antecedent than 
consequence components, although implemen- 
tation was variable in some cases and at lower 
rates than desirable. In the absence of perfor- 
mance feedback, teachers may have more 
successfully implemented antecedent compo- 
nents because these interventions were similar 
across students. For example, in Classroom 2, 3 
students had daily scheduling with preferred 
following nonpreferred activities and choices 
built into all tasks, 2 students needed demands 
to be presented in individualized ways, and 2 
students needed transition warnings. As a result, 
these antecedent procedures may have operated 
as part of the daily routine. In addition, the 
complexity (e.g., various schedules and types of 
reinforcement) and number of consequence 
procedures may account for lower treatment 
integrity of consequence components in base- 
line. Which consequence components were 
more likely to be correctly employed (e.g., 
positive vs. negative reinforcement) was not 
examined; however, research has previously 
demonstrated that teachers tend to focus their 
attention on students’ inappropriate behavior 
(Cooper, Thomson, & Baer, 1970). Future res- 
earch might consider assessing teachers’ imple- 
mentation of specific consequence components. 

Differences in implementation of antecedent 
and consequence procedures evident in this 
study may have interesting implications for the 
types of interventions used to increase teachers’ 
treatment integrity. These findings suggest that 
future studies should continue to examine the 
integrity of individual treatment components 
(Gresham et al., 1993). Analysis of whether 
treatment integrity is a function of teacher 
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skill or motivational deficits is also a worthy 
topic for investigation. For example, exploring 
whether treatment integrity increases as a func- 
tion of performance feedback, teachers’ role in 
intervention planning, contingent reinforce- 
ment, training, or a combination of these may 
begin to address whether treatment integrity is 
a result of a skill or motivational deficit. 

Despite its documented success, the potency 
of performance feedback has been criticized for 
several reasons, including the following: (a) 
Results have not been maintained at high rates 
(Mortenson & Witt, 1998; Witt et ah, 1997). 
(b) It may be time consuming (Mortenson & 
Witt, 1998). (c) There may be a lack of 
acceptability by change agents (e.g., teachers) 
(Mortenson & Witt, 1998) as well as (d) 
reactivity to observers (Mortenson & Witt, 
1998; Witt et ah, 1997). To increase the 
sophistication of treatment-integrity interven- 
tions and specifically performance feedback, 
attempts to address these concerns were 
included in the current study. 

Improvements in the integrity of behavior 
support plan implementation were maintained 
in the current study. This may be due to 
the length of time performance feedback 
was employed and the latency between ses- 
sions. Previous research implemented perfor- 
mance feedback for approximately 6 weeks 
(Mortenson & Witt, 1998; Noell et ah, 2000) 
compared to 8 to 22 weeks in our study. 
Whereas other studies utilized daily (Noell 
et ah, 1997, 2002; Witt et ah, 1997) or weekly 
(Mortenson & Witt, 1998) performance feed- 
back, we provided feedback every other week. 
Important to note is that performance feedback 
was provided during the same days that direct 
observations occurred. Consequently, recent 
examples of teachers’ behavior could be 
reviewed during feedback sessions. This sched- 
ule of performance feedback may lead to the 
maintenance of treatment effects in the short 
term and is time efficient and practical for 
clinicians to implement. However, it is unlikely 


that these results would be maintained over the 
long term. We suspect that periodic collection 
of treatment integrity data and subsequent 
performance feedback are necessary for high 
rates of intervention integrity to persist. 

The importance of investigating the social 
validity of performance feedback has evolved 
from suggestions (Mortenson & Witt, 1998) that 
feedback may serve as a negatively reinforc- 
ing event for some teachers. All of the teachers in 
the current study rated performance feedback 
favorably with respect to the purpose, procedures, 
and outcome; this is consistent with the findings 
of Noell et al. (2002). Assessing the acceptability 
of performance feedback provides a prelim- 
inary attempt to identify how teachers per- 
ceive treatment integrity interventions. Future 
researchers may consider experimentally analyz- 
ing the function that performance feedback 
serves for teachers, rather than relying on verbal 
report. 

The current investigation has several limita- 
tions. First, it is possible that treatment effects 
were confounded by reactivity to being 
observed. Consistent with recommendations 
by Gresham et al. (1993), we attempted to 
reduce reactivity by conducting observations on 
a variable-time schedule and used consultants 
who had been assigned to these classrooms as 
primary observers. Therefore, it was common 
for teachers to see these consultants in the 
classroom conducting observations. Although 
we attempted to make these observations un- 
obtrusive, teachers were aware during the 
intervention phase that they were being 
observed. A second limitation of the study is 
that implementation of performance feedback 
occurred when either antecedent or conse- 
quence components were stable or decreasing. 
As a result, experimental control may not be 
as robust for Ms. Malden, whose correct 
implementation of consequence components 
appeared to be improving in baseline. Every 
attempt was made to have stable or decreasing 
trends in baseline prior to feedback, but 
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a clinical decision was made to employ 
performance feedback when the implementa- 
tion of antecedent components was stable. 
Third, interobserver agreement was based on 
consistency between consultants’ ratings on 
each plan component, but a more conservative 
assessment of agreement would have been to 
assess whether each consultant recorded the 
same type and number of deviations from the 
plan component in proportion to the number 
of opportunities the teacher had to implement 
the component. Fourth, it should be noted that 
verbal report was used to assess the social 
validity of performance feedback; this may be 
subject to rater bias given the frequency with 
which teachers worked with the consultants. 
Fifth, the functions of individual components 
included in performance feedback were not 
analyzed. For example, praise and corrective 
feedback may be differentially effective. Sixth, 
we categorized interventions as antecedent and 
consequence procedures. Although an attempt 
was made to use the literature available to make 
these determinations, some procedures (e.g., 
providing breaks from work and self-monitor- 
ing) could be categorized as either antecedent or 
consequence procedures. It may be fruitful to 
explore the classification of interventions con- 
sidered to be antecedents and consequences. 
Finally, the impact of increased treatment 
integrity on students’ maladaptive behavior 
was not examined. Ultimately, the interest in 
developing procedures to increase treatment 
integrity has resulted from its impact on 
treatment efficacy. Therefore future studies 
should make an effort to provide corresponding 
information on students’ behavior. 

In summary, the results of this study suggest 
that performance feedback is an effective and 
acceptable intervention for increasing the 
treatment integrity with which special educa- 
tion teachers administer antecedent and conse- 
quence components in the context of an 
ongoing behavior support plan. Although the 
length of time this intervention was employed 


extends that of previous performance feedback 
studies, feedback was provided for an average of 
12 min every other week, which is both 
practical and time efficient. This study also 
introduced a method for using direct observa- 
tion procedures instead of permanent products 
to assess treatment integrity. 
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APPENDIX 


Integrity Assessment for Behavior Support Plans 

Date: Student: Brian 

Staff: Ms. Lowell Observer: 


Time Start: Time End: 


Type of 
Procedure 

Description of Plan Components 

(Indicate which of the following 
procedures are part of the BSP by 
placing a V in the box) 

Implementation Rating 

(Indicate the level of 
integrity with which staff are 
implementing the procedure) 

Comments/Examples 
(Provide an example of 
how the staff 
implemented the 
procedure) 

Antecedent 

□ Transition Procedures 

Prior to change in activities 
give 3 warnings (2m, lm, 30s) 

If there is a time limit to 
activities, review this verbally 
prior to activity & use a timer 
to indicate when activity has 
ended 

When activity is room change, 
preview what Brian needs to 
do when he gets to location 
PRIOR to changing rooms 

□ Implemented as Written 
(every time) 

□ NOT Implemented as 
Written (sometimes or 
never as written) 

□ No opportunity to observe 
(teacher had no 
opportunity to exhibit) 



□ Schedule 

Brian creates visual schedule 2 
times daily (9 AM & 12:30 

PM) 

Schedule preferred activities 
following non-preferred 

Review (tell) changes in 
schedule to Brian immediately 

□ Implemented as Written 
(every time) 

□ NOT Implemented as 
Written (sometimes or 
never as written) 

□ No opportunity to observe 
(teacher had no 
opportunity to exhibit) 



□ Choice 

- Give choice of 2 activities at 
all times even during preferred 
activities 

- Never terminate an activity 
w/o providing choice of next 
activity 

□ Implemented as Written 
(every time) 

□ NOT Implemented as 
Written (sometimes or 
never as written) 

□ No opportunity to observe 
(teacher had no 
opportunity to exhibit) 
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□ Pre-teaching Seatwork 

- Let Brian know if he gets 
frustrated he needs to raise 
his hand and ask for help 

- When expected to work 
independently, only give 
programs with an T on top 
(indicating he can do with 
80% or better accuracy) 
Work w/o “I” = teachers 
sit and provide specific 
directions, model, and help 
him work through problems 

□ Implemented as Written 
(every time) 

□ NOT Implemented as 

Written (sometimes, 
partially, or never as 
written) 

□ No opportunity to observe 
(teacher had no 
opportunity to exhibit) 



□ Seating/Physical Proximity 
- Desk located in area where 
distractions are limited 

Not seated within arms 
length of students or 
teachers 

□ Implemented as Written 
(every time) 

□ NOT Implemented as 

Written (sometimes, 
partially, or never as 
written) 

□ No opportunity to observe 
(teacher had no 
opportunity to exhibit) 



□ Demand Presentation 

- Provide specific statements 
(i.e., first-then) 

- Establish eye contact first 

- Use neutral tone of voice 

- Divide demands & tasks 
into small parts 

□ Implemented as Written 
(every time) 

□ NOT Implemented as 

Written (sometimes, 
partially, or never as 
written) 

□ No opportunity to observe 
(teacher had no 
opportunity to exhibit) 


Consequences 

□ Differential Reinforcement 
(attention) 

- If Brian engages in 
appropriate social 
interaction w / peers & 
teachers provide specific 
praise and attention 

VI 10 min give attention 
and praise if engaged in 
appropriate other behavior 

□ Implemented as Written 
(every' time) 

□ NOT Implemented as 

Written (sometimes, 
partially, or never as 
written) 

□ No opportunity to observe 
(teacher had no 
opportunity to exhibit) 
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□ Differential 

Reinforcement (lottery) 
Provide lottery ticket if 
Brian engages in 
appropriate social 
interaction with peers 
and when engaged in 
appropriate behavior 
during group 

□ Implemented as 

Written (every time) 

□ NOT Implemented as 
Written (sometimes, 
partially, or never as 
written) 

□ No opportunity to 
observe (teacher had no 

opportunity to exhibit) 



□ Planned Ignoring 
- No mention of, eye 
contact, or other 
attention given to 
inappropriate sexual 
behavior 

Reinforce (specific 
praise) others in 
proximity of Brian for 
engaging in appropriate 
behavior 

□ Implemented as 

Written (every time) 

□ NOT Implemented as 
Written (sometimes, 
partially, or never as 
written) 

□ No opportunity to 
observe (teacher had no 

opportunity to exhibit) 



□ Block and Redirect 

- Block 

aggression/destruction 

- Redirect to desk and 
current activity 

- After completes original 
demand, tell Brian to 
restore damage 

□ Implemented as 

Written (every time) 

□ NOT Implemented as 
Written (sometimes, 
partially, or never as 
written) 

□ No opportunity to 
observe (teacher had no 

opportunity to exhibit) 



□ Response Cost 

Starts day w/ 10 pennies 
- If engaged in peer 
instigation or 
inappropriate speech 
(i.e., swearing, backtalk, 
demanding, yelling) 

1 verbal warning 

If occurs a 2 nd time 
within 5 min, take penny 

□ Implemented as 

Written (every time) 

□ NOT Implemented as 
Written (sometimes, 
partially, or never as 
written) 

□ No opportunity to 
observe (teacher had no 

opportunity to exhibit) 



Total components in the behavior support plan: 1 1 

Total plan components teachers had the opportunity to exhibit: 

Antecedent plan components teachers had the opportunity to exhibit: _ 

Antecedent plan components teachers did not have the opportunity to exhibit: _ 

Antecedent plan components teachers had the opportunity to exhibit: 

where implementation was as written: %_ 

where implementation was not as written: %_ 

Consequence plan components teachers had the opportunity to exhibit: 

Consequence plan components teachers did not have the opportunity to exhibit: 

Consequence plan components teachers had the opportunity to exhibit: 

where implementation was as written: %. 

where implementation was not as written: %_ 
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STUDY QUESTIONS 


1. What are the typical components of a performance feedback system? 

2. How did the authors attempt to minimize observer reactivity? 

3. Briefly summarize the effects observed during the baseline and feedback conditions. 

4. Given that performance feedback contains a number of elements, what types of behavioral processes 
may have accounted for the observed results? 

5. To what did the authors attribute higher levels of treatment integrity for antecedent versus 
consequence procedures during baseline? 

6. What type of data, not collected in the study, might have strengthened the authors’ conclusions 
about improvements in teacher performance? 

7. The authors discussed treatment integrity failures as a function of deficits related to skills versus 
motivation. Why might feedback as an intervention be unable to distinguish between the two types 
of deficits? 


8. What are some potential advantages and disadvantages of biweekly observation and performance 
feedback? 

Questions prepared by Natalie Rolider and Sarah E. Bloom, University of Florida 


